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Claims: 

1 . A system for executing a multimodal software application, comprising: 

the multimodal software application, wherein said multimodal software 
application is configured to receive first data input from a first set of peripheral 
devices and output second data to a second set of peripheral devices; 
5 a dialog engine in communication with the multimodal software application, 

wherein said dialog engine is configured to execute a workflow description related 
to the multimodal software application and provide the first data to the multimodal 
software application; and 

a respective interface component associated with each peripheral device 
1 0 within said first and second sets; wherein each interface component is configured 
to provide the second data, if any, to the associated peripheral device and receive 
the first data, if any, from the associated peripheral device. 
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2. The system according to claim 1 , wherein a peripheral device can be a member 
of both the first and second sets. 

3. The system according to claim 1, wherein the first set of peripheral devices 
includes a speech synthesizer and the second set of peripheral devices includes 
a speech recognizer. 

4. The system according to claim 1 , wherein the multimodal software application 
further comprises a graphical user interface including a screen. 

5. The system according to claim 4, wherein the workflow description comprises 
a set of workflow objects, wherein each workflow object is associated with a 
respective visual control within said screen. 

6. The system according to claim 5, wherein each workflow object further 
comprises: 

a prompt related to the associated visual control; and 
a link to another workflow object. 

7. The system according to claim 6, wherein each workflow object further 
comprises: 

a plurality of expected input values; and 
a help message. 
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8. The system according to claim 6, wherein each workflow object further 
comprises: 

a first identification of members of the first set from which first data can be 
received; and a second indication of members of the second set to which second 
data can be sent. 

9. The system according to claim 6, wherein the prompt is the second data. 

10. The system according to claim 1, wherein the dialog engine is further 
configured to redirect the first data to a third set of peripheral devices comprising 
selected members from the first and second set. 

1 1 . The system according to claim 6, wherein each workflow object further 
comprises: 

a plurality of links, each link being to a different respective workflow object 
and each of the plurality of links having an activation criterion. 

12. The system according to claim 1 1 , wherein the activation criterion relates to 
a value of the first data. 
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13. The system according to claim 6. wherein the dialog engine is further 
configured to: 

execute a particular workflow object by outputting the prompt as the second 

data; 

instruct each interface component associated with a respective member of 
the first set to wait for the first data; 

based on the first data determine whether to follow the link; and 
execute said another workflow object. 

14. The system according to claim 6, wherein the dialog engine is further 
configured to: 

receive the first data; and 

foHA^ard the first data to each interface component associated with a 
respective member of the second set. 

15. The system of claim 4, wherein the execution of the workflow description is 
synchronized to a display, by the dialog engine, of the graphical user interface. 
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16. A system for executing a multimodal software application comprising: 

a dialog engine in communication with a) the multimodal software 
application, b) a first set of peripheral devices for receiving first data, and c) a 
second set of peripheral devices for outputting second data; and 

said dialog engine configured to execute a workflow description related to 
the multimodal software application, wherein executing the workflow description 
includes generating the second data from the workflow description and providing 
the first data to the multimodal software application. 
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1 7. The system according to claim 16. wherein the first set of peripheral devices 
includes a speech synthesizer and the second set of peripheral devices includes 
a speech recognizer. 

18. The system according to claim 16, wherein the dialog engine is configured to 
communicate with each of the peripheral devices in the first and second sets via 
a respective interface component associated with each peripheral device within 
said first and second set; wherein the interface component is configured to provide 

5 the second data, if any, to the associated peripheral device and receive the first 
data, if any, from the associated peripheral device. 

19. The system according to claim 16, wherein the multimodal software 
application further comprises a graphical user interface including a screen. 

20. The system according to claim 1 9, wherein the workflow description comprises 
a set of workflow objects, wherein each workflow object is associated with a 
respective visual control within said screen. 

21. The system according to claim 20, wherein each workflow object further 
comprises: 

a prompt related to the associated visual control; and 
a link to another workflow object. 
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22. The system according to claim 21. wherein the dialog engine further 
comprises: 

an execution unit configured to execute a particular workflow object; 

a prompt generator configured to output the prompt as the second data; 
5 a data tester configured to determine if the first data received in response 

to the second data satisfies a set of criteria associated with the link; and 

an object loader configured to load the another workflow object In the 
execution unit when instructed by said data tester. 
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23. A method for developing multimodal software applications, said method 
comprises the steps of: 

a) receiving a portion of code implementing a first visual control within a 
screen of a graphical user interface; 
5 b) generating a corresponding dialog unit based on the portion of code; and 

c) creating a link between the corresponding dialog unit and another dialog 
unit associated with a second visual control within the screen. 
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24. The method according to claim 23, wherein the step of generating the 
corresponding dialog unit includes the steps of: 

extracting an expected set of inputs from the portion of code to populate the 
dialog unit. 

25. The method according to claim 23, wherein the step of generating the 
corresponding dialog unit includes the steps of: 

extracting a default prompt from the portion of code to populate the dialog 

unit. 

26. The method according to claim 23, wherein the step of generating the 
corresponding dialog unit includes the step of: 

extracting a default help prompt from the portion of code to populate the 
dialog unit. 

27. The method according to claim 24, further comprising the step of: 

creating a default help prompt based on the expected set of inputs. 

28. The method according to claim 23, further comprising the steps of: 

receiving modification input for the dialog unit; and 
modifying the dialog unit in accordance with the input. 

29. The method according to claim 23, wherein the dialog unit comprises: 

a prompt for outputting to one or more peripheral devices; 
the link. 
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30. The method according to claim 29, wherein the dialog unit further comprises: 

a first identification of the one or more peripheral devices; and 
a second identification of a set of peripheral devices from which to receive 
first data in response to the prompt. 

31 . The method according to claim 23, further comprising the steps of: 

repeating the steps a) through c) for a plurality of visual control elements 
within the screen; and 

combining the resulting plurality of dialog units and links into a workflow 
description corresponding to the screen. 

32. The method according to claim 23, further comprising the steps of: 

identifying a set of previously generated dialog units and previously created 
links; and 

storing the set as a reusable object. 

33. The method according to claim 32, further comprising the step of: 

retrieving the reusable object to generate the corresponding dialog unit. 
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34. A method for executing a multimodal software application having a graphical 
user interface with a screen, the method comprising the steps of: 

receiving a workflow description corresponding to the screen, and 
executing the workflow description in synchronization with the graphical 
user interface. 
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35. The method according to claim 34, wherein the step of executing includes the 
steps of: 

identifying a workflow object associated with a visual control on the screen; 
executing the workflow object; and 

identifying another workflow object linked to the workflow object. 

36. The method according to claim 35, wherein the step of executing the workflow 
object includes the steps of: 

extracting a prompt from the workflow object; 

sending the prompt to a first set of peripheral devices; 

instructing a second set of peripheral devices to wait for a response; and 

forwarding the response to the multimodal software application. 

37. The method according to claim 36, further comprising the step of: 

forwarding the response to one or more of the peripheral devices of the first 
and second sets. 

38. The method according to claim 36, further comprising the steps of: 

extracting a default help prompt from the workflow object; and 
sending the default help prompt to one or more of the peripheral devices of 
the first and second sets. 



-42- 

39. The method according to claim 34. wherein the step of executing includes the 
steps of: 

outputting audio prompts corresponding to the screen; and 
receiving input via speech recognition system in response to the audio 
prompts. 

40. The method according to claim 36, further comprising the steps of: 

extracting information regarding peripheral device membership in the first 
and second sets from the workflow object. 

41 . The method according to claim 36, wherein the response also includes data 
related to the another workflow object. 
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42. A system for developing a multimodal application comprising: 

a code extractor configured to analyze a portion of code implementing a 

visual control within a screen of a graphical user interface; 

a dialog creator, in communication with the code extractor, configured to 

generate a workflow object based on the analysis of the portion of code; and 

a linker configured to generate a link to another workflow object, said link 

being a portion of the workflow object. 
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43. The system according to claim 42, wherein the code extractor is further 
configured to populate a prompt based on parameters of the visual control. 

44. The system according to claim 43, wherein the code extractor is further 
configured to: 

identify a set of expected inputs, if any, to the prompt; and 
identify a default help prompt. 

45. The system according to claim 42 further comprising: 

a library of predefined workflow objects, each relating to a plurality of visual 
controls within said graphical user interface; and 

an object retriever configured to preempt the dialog creator and generate 
the workflow object by extracting one of the predefined workflow objects from the 
library. 

46. The system according to claim 42, wherein the dialog creator further 
comprises: 

an editor configured to receive input and modify the workflow object in 
accordance with the input. 



-45- 

47. The system according to claim 42, further comprising: 

a controller configured to manage the operation of the code extractor, the 
dialog generator, and the linker with respect to each visual control on the screen 
so as to generate a corresponding workflow object for each visual control; and 

a workflow creator configured to combine the workflow objects into a 
workflow description. 
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48. A computer-readable medium bearing instructions for executing a multimodal 
software application having a graphical user interface with a screen, said 
instructions being arranged, upon execution thereof, to cause one or more 
processors to perform the steps of: 

receiving a workflow description corresponding to the screen, and 
executing the workflow description in synchronization with the graphical 
user interface. 

49. A computer-readable medium bearing instructions for developing multimodal 
software applications, said instructions being arranged, upon execution thereof, 
to cause one or more processors to perform the steps of: 

receiving a portion of code implementing a first visual control within a 

screen of a graphical user interface; 

generating a corresponding dialog unit based on the portion of code; and 
creating a link between the corresponding dialog unit and another dialog 

unit associated with a second visual control within the screen. 



